Glaciers

View a running version of this notebook. | Download this project.


Glaciers explorer using Datashader

This notebook provides an annotated HoloViews+Panel implementation of a dashboard originally developed in Plotly+Dash for viewing data about the Earth's glaciers from the Open Global Glacier Model. To run it, first:

conda install -c pyviz pandas=0.24 param=1.10.0 panel=0.10.1 holoviews=1.13.5 datashader=0.11.1

Next, save the data file as data/oggm_glacier_explorer.csv (and gzip it if desired).

The dashboard can then be used here as a cell in the Jupyter notebook, or you can run it as a separate server using:

panel serve glaciers.ipynb --show

This notebook is essentially the same as Glaciers.ipynb but uses unaggregated data that is practical only with Datashader.

In [1]:
import numpy as np
import pandas as pd
import holoviews as hv
import datashader as ds
import panel as pn

from colorcet import bmy
from holoviews.operation.datashader import rasterize, datashade

hv.extension('bokeh')

Load the data

Here we will load the glaciers data and project the latitudes and longitudes to Google Mercator coordinates, which will allow us to plot it on top of a tile source. We use the pn.state.as_cached function to cache the data to ensure that only the first visitor to our app has to load the data.

In [2]:
def load_data():
    df = pd.read_csv('data/oggm_glacier_explorer.csv')
    df['latdeg'] = df.cenlat
    df['x'], df['y'] = ds.utils.lnglat_to_meters(df.cenlon, df.cenlat)
    return df

df = pn.state.as_cached('glaciers', load_data)

df.tail()
Out[2]:
rgi_id cenlon cenlat area_km2 glacier_type terminus_type mean_elev max_elev min_elev avg_temp_at_mean_elev avg_prcp latdeg x y
213745 RGI60-18.03533 170.354 -43.4215 0.189 Glacier Land-terminating 1704.858276 2102.0 1231.0 2.992555 6277.991881 -43.4215 1.896372e+07 -5.376350e+06
213746 RGI60-18.03534 170.349 -43.4550 0.040 Glacier Land-terminating 2105.564209 2261.0 1906.0 0.502311 6274.274146 -43.4550 1.896316e+07 -5.381486e+06
213747 RGI60-18.03535 170.351 -43.4400 0.184 Glacier Land-terminating 1999.645874 2270.0 1693.0 1.187901 6274.274146 -43.4400 1.896339e+07 -5.379186e+06
213748 RGI60-18.03536 170.364 -43.4106 0.111 Glacier Land-terminating 1812.489014 1943.0 1597.0 2.392771 6154.064456 -43.4106 1.896483e+07 -5.374680e+06
213749 RGI60-18.03537 170.323 -43.3829 0.085 Glacier Land-terminating 1887.771484 1991.0 1785.0 1.351039 6890.991816 -43.3829 1.896027e+07 -5.370436e+06

Plot the data

As you can see in the dataframe, there are a lot of things that could be plotted about this dataset, but following the previous version let's focus on the lat/lon location, elevation, temperature, and precipitation. We'll use tools from HoloViz, starting with HoloViews as an easy way to build interactive Bokeh plots. So that we can use the full glacier database with good performance, we'll have Datashader pre-render some of the plots as images before they reach the browser.

To start, let's declare a HoloViews object that captures English-text descriptions of the various columns in the dataframe, in a way that subsequent plots can all inherit without having to repeat that information:

In [3]:
data = hv.Dataset(df, [('x', 'Longitude'), ('y', 'Latitude')],
                     [('avg_prcp', 'Annual Precipitation (mm/yr)'),
                      ('area_km2', 'Area'), ('latdeg', 'Latitude (deg)'),
                      ('avg_temp_at_mean_elev', 'Annual Temperature at avg. altitude'), 
                      ('mean_elev', 'Elevation')])
total_area = df.area_km2.sum()

print(data, len(data), total_area)
:Dataset   [x,y]   (avg_prcp,area_km2,latdeg,avg_temp_at_mean_elev,mean_elev) 213750 613225.6620000001

Here we've declared that x and y (the projected lat,lon location of the center of the glacier) are the "key dimensions" (independent values that specify which glacier this is), and the rest are "value dimensions" (various dependent values characterizing that particular sample).

Next, let's define various options that will control the appearance of our plots:

In [4]:
geo_kw    = dict(aggregator=ds.sum('area_km2'), x_sampling=1000, y_sampling=1000)
elev_kw   = dict(cmap='#7d3c98')
temp_kw   = dict(num_bins=50, adjoin=False, normed=False, bin_range=data.range('avg_temp_at_mean_elev'))
prcp_kw   = dict(num_bins=50, adjoin=False, normed=False, bin_range=data.range('avg_prcp'))

size_opts = dict(min_height=400, min_width=600, responsive=True)
geo_opts  = dict(size_opts, cmap=bmy, logz=True, colorbar=True, xlabel='', ylabel='')
elev_opts = dict(size_opts, show_grid=True)
temp_opts = dict(size_opts, fill_color='#f1948a', default_tools=[], toolbar=None, alpha=1.0)
prcp_opts = dict(size_opts, fill_color='#85c1e9', default_tools=[], toolbar=None, alpha=1.0)

Using these options with HoloViews, we can plot various combinations of the variables of interest:

In [5]:
geo_bg = hv.element.tiles.EsriImagery().opts(alpha=0.6, bgcolor="black")
geopoints = hv.Points(data, vdims=['area_km2']).opts(**geo_opts)

(geo_bg*rasterize(geopoints, **geo_kw).options(**geo_opts) + 
 datashade(data.to(hv.Scatter, 'mean_elev','latdeg', []), **elev_kw).options(**elev_opts) + 
 data.hist('avg_temp_at_mean_elev', **temp_kw).options(**temp_opts) +
 data.hist('avg_prcp',              **prcp_kw).options(**prcp_opts)).cols(2)
Out[5]:

In the top left we've overlaid the location centers on a web-based map of the Earth, separately making a scatterplot of those same datapoints in the top right with elevation versus latitude. The bottom rows show histograms of temperature and precipitation for the whole set of glaciers. Of course, these are just some of the many plots that could be constructed from this data; see holoviews.org for inspiration.

Define plotting functions

The above plots are useful for understanding the properties of all glaciers worldwide, but what's more interesting is to consider how some particular subset of the glaciers relates to the rest. To explore this, let's capture the above commands into some functions that will accept a dataset and return viewable plots for that particular data. That way we can plot selected subsets of the data and compare them to the plots of the full dataset.

In [6]:
def geo(data):   return hv.Points(data).options(alpha=1)
def elev(data):  return data.to(hv.Scatter, 'mean_elev', 'latdeg', [])
def temp(data):  return data.hist('avg_temp_at_mean_elev', **temp_kw).options(**temp_opts)
def prcp(data):  return data.hist('avg_prcp',              **prcp_kw).options(**prcp_opts)

If called with the full dataset:

(geo_bg*rasterize(geo(data), **geo_kw).options(**geo_opts) + datashade(elev(data), **elev_kw).options(**elev_opts) + temp(data) + prcp(data)).cols(2)

these functions will return static plots just like those above. Let's capture that output as a set of low-opacity (alpha<0.5) plots to use as a background on which to show selected subsets of the data:

In [7]:
static_geo  = rasterize(geo(data),   **geo_kw).options(alpha=0.1, tools=['hover'], active_tools=['box_select'], **geo_opts)
static_elev = datashade(elev(data), **elev_kw).options(alpha=0.1, active_tools=['box_select'], **elev_opts)
static_temp = temp(data).options(alpha=0.1)
static_prcp = prcp(data).options(alpha=0.1)

Here we defined some Bokeh tools like hover and box_select that you'll see below. Meanwhile, we could plot these on their own if we wished:

(geo_bg*static_geo + static_elev + static_temp + static_prcp).cols(2)

Add linked selections

All we have to do to add linked selections to these static plots is make a hv.link_selections instance and apply it to our plots:

In [8]:
ls = hv.link_selections.instance()

geomap = geo_bg * ls(static_geo)
elevation = ls(static_elev)
temperature = ls(static_temp)
precipitation = ls(static_prcp)

(geomap + elevation + temperature + precipitation).cols(2)
Out[8]:

Let's also create a pane that renders the count of total selections:

In [9]:
def count(data): 
    selected_area  = np.sum(data['area_km2'])
    selected_percentage = selected_area / total_area * 100
    return pn.pane.Markdown(
        '## Glaciers selected: {} | Area: {:.0f} km² ({:.1f}%)</font>'.format(
            len(data), selected_area, selected_percentage),
        align='center', width=500
    )

dynamic_count = pn.bind(count, ls.selection_param(df))
pn.panel(dynamic_count)
Out[9]:

Dashboard

The code and plots above should be fine for exploring this data in a notebook, but let's go further and make a shareable dashboard using Panel. Panel lets us add arbitrary custom functionality, such as a button to reset the selections by calling clear_selections which sets the selection_expr to None:

In [10]:
pn.extension()

def clear_selections(event):
    ls.selection_expr = None

clear_button = pn.widgets.Button(name='Clear selection', align='center')
clear_button.param.watch(clear_selections, 'clicks');

And we can add static text, Markdown, or HTML items like a title, instructions, and logos:

In [11]:
title       = '<div style="font-size:35px">World glaciers explorer</div>'
instruction = 'Box-select on each plot to subselect; clear selection to reset.<br>' + \
              'See the <a href="https://github.com/panel-demos/glaciers">Jupyter notebook</a> source code for how to build apps like this!'
oggm_url    = 'https://raw.githubusercontent.com/OGGM/oggm/master/docs/_static/logos/oggm_s_alpha.png'
oggm_logo   = '<a href="https://oggm.org"><img src="{0}" width=170></a>'.format(oggm_url)
pn_url      = 'http://panel.pyviz.org/_static/logo_stacked.png'
pn_logo     = '<a href="https://panel.pyviz.org"><img src="{0}" width=140></a>'.format(pn_url)

If you want detailed control over the formatting, you could define these items in a separate Jinja2 template. But here, let's put it all together using Panel Row and Column objects, which can display objects and plots from many different libraries, including the HoloViews objects used here. You'll then get an app with widgets and plots usable from within the notebook:

In [12]:
header = pn.Row(pn.Pane(oggm_logo),  pn.layout.Spacer(width=30), 
                pn.Column(pn.Pane(title, width=400), pn.Pane(instruction, width=500)),
                pn.layout.HSpacer(), pn.Column(dynamic_count, pn.layout.Spacer(height=20), clear_button), 
                pn.Pane(pn_logo, width=140))

content = pn.Column(header, pn.Row(geomap, elevation), pn.Row(temperature, precipitation), width_policy='max', height_policy='max')

content
Out[12]:

Lastly we will build a Template to give this dashboard a more polished look and feel when deployed, reflecting the image shown at the top of the notebook:

In [13]:
template = pn.template.MaterialTemplate(title='World Glaciers Explorer')

template.header.append(
    pn.Row(
        pn.layout.HSpacer(),
        dynamic_count,
        clear_button,
    )
)

template.sidebar.extend([
    pn.pane.PNG(oggm_url, width=250),
    pn.pane.Markdown(instruction, width=250, height=100),
    pn.pane.PNG(pn_url, width=200, align='center')
])

template.main.append(
    (geomap + elevation + temperature + precipitation).cols(2)
)

template.servable();

As long as you are running this notebook "live" (in Jupyter, not viewing a website or a static copy on anaconda.org), the above notebook cell should contain the fully operational dashboard here in the notebook. You can also launch the dashboard at a separate port that shows up in a new browser tab, either by changing .servable() to .show() above and re-executing that cell, or by leaving the cell as it is and running panel serve --show glaciers.ipynb.

Either way, you should get a standalone dashboard like the image at the start of this notebook. You can now select and explore your data to your heart's content, and share it with anyone else interested in this topic! Or you can use the above approach to make your own custom dashboard for just about anything you want to visualize, with plots from just about any plotting library and arbitrary custom interactivity for libraries that support it.

This web page was generated from a Jupyter notebook and not all interactivity will work on this website. Right click to download and run locally for full Python-backed interactivity.

View a running version of this notebook. | Download this project.